Are Better Feature Selection Methods Actually Better? - Discussion, Reasoning and Examples

One of the hot topics discussed recently in relation to pattern recognition techniques is the question of actual performance of modern feature selection methods. Feature selection has been a highly active area of research in recent years due to its potential to improve both the performance and economy of automatic decision systems in various applicational fields, with medical diagnosis being among the most prominent. Feature selection may also improve the performance of classifiers learned from limited data, or contribute to model interpretability. The number of available methods and methodologies has grown rapidly while promising important improvements. Yet recently many authors put this development in question, claiming that simpler older tools are actually better than complex modern ones – which, despite promises, are claimed to actually fail in real-world applications. We investigate this question, show several illustrative examples and draw several conclusions and recommendations regarding feature selection methods’ expectable performance.